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and senior high school students. A randomly ^elected group of junior 
and senior high school science teachers and randomly selected cJLasses 
of theirs were administered achievement tests in science. Chi square 
and analysis of coyariance (A'NCbVA) jvere used to analyze the data. 
Participation in NSF Science jliistitutes and geographic regions were 
factors in a t¥Orway ANCOVA. jClass liean for achievement was the 
dependent vatiable and teachet achievement was the coyafiate. NSF 
participation was found to be| a significant factor in student 
achievement for high school teachers but not for junior high school 
teachers. The study was replicated for mathematics teachers and' 
classes and the same results were found.. It was concluded that ^ 
significance at the senior high\sqhool and nonsignif icance at the 
junipr high level may be due to\the difference in subject matter 
taught and the population of students between the two levels. 
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A randomly selected group of junior high school and senior 
high school science teachers and randomly selected classes of 
their.s were administered; achievement tests in science. Partici- '. 
pation in NSF. Science Institutes and geographic region were 
factors in a two-way ANCOVA. Class mean for achievement was the' • 
dependent variable and' teacher achievement was the covariate. 
NSF participation was found to be a significant factor' in student 
achievement for senior high school teachers but not for junior 
high school teachers ji The study was replicated for mathematics 
teachers and classes and the same results were found. 

Large foundations and institutes across the nation such as the National 
Science Foundation (NSF) have often as sumied that the' improvement of teacher, 
knowledge and ability will result in improved student achievement. As a result 
of this assumption; much money and effort has been devoted to improving 
teacher knowledge, particularly in science. Since 1958, the NSF has .spent, 
nearly 750 million .dollars for teacher training and upgrading. Most of the 
money 'has been used to support various training institutes, such as the Academic 
Year Institute (AYI), In-Service Institutes (ISI) , Sequehtia;* Summer Institutes 
(SSI), and Unitary Summer Institutes (USI) . Little evidence has been provided 
to show that the ultimate consumers, school children;, have benefited! This 
report provides some additional evidence of benefit. 



Previous Studies 

A review -^f follow-up studies in Academic Year ^vistitutes and In-Service 
teacher training Un. the areas of science and mathematics gives mixed results, 
about improvement in the' quality of education within these subject areas for 
teachers and students. In the. utilization of 'the American Association for 
the Advancement of Science's (AA^AS) "Science— A Process Approach** (SAPA) and 



the "Science Curriculum Improvement Study" "(SCIS) Curricula j results are pre- 
sented that teachers demonstrated marked improvement in^the nine competency areas 
considered (Lashier and Kurtz,. 1970) as well as in the desire to. initiate 

science'curriculum change through the use of the AAAS and SCIS philosophies 

■ ■ / 

and activities (Merkle, 1970). Moreover , ^Lashier and Kurtz (1970), in an. 
experiment' conducted after a summer institute, reported that students using 
the AAAS method scored significantly- higher on three of . four competency. ■ 
tasks than did a control group using other .materials . Studies involving 
Academic Year. Institutes participants also indicate that . teachers 
experienced wider opportunities to us^ their abilities and improved their 
professional and economic stature. (Bradberry, 1967; Irby, 1967; Martinen, 

1967; Westmeyer, et.al., 1967). 

■ " A . ; . ■ \ ■ . ' - 

In research reported involving Mathematics Academic Year Institutes, 

the Montgomery County (Md.) Public Schools. held a three-week summer 

^laboratory .workshop in 1969 for teachers and low achievers in mathematics 

at the secondary level. The students were given a pre- and post-test 

sequence of instruments measuring mathematics achievement and self -concepts . - 

The results of all tests indicated that (1) students showkd an average gain 

of about one-half year in Mathematics Achievement, (2) that their self- 

concepts with regard to mathematics increased significantly ,/and that (3) 

none of the students' comments suggested -unfavorable dispositions to the 

workshop as a whole. Regression effects may account for most of these 

changes, however. This study is one of only a few examining mathematics 

learning related to institute participation . (See Irby, 1967; Martinen, 1967; ■ 

Krieghbaum, 1968; and Slawson, 1970). . c 



In an internatiou . o * ?.dy of science curricula In nineteen countries, 

Comber and Keeves (1973) examined the relationships between teacher-related 

variables and student- ac'ilevement. They reported 

There Is so^t,?i evidence th'kt teachers who show a 
professlon&A Vttltude toward their work, for example. 
In the preparation they do and In their ;membershlp 
In professional associations,* achieve, In general, 
better results than those who do not. 

Specific variables reported significant, in regression analyses on achieve- 
ment were teacher training In six of fifteen countries at the senior high level 
and attendance at conferences in two countries at the- senior high level. 

Thelen .and^Lltskey (1972) reported ^^^u-sXudy in which students of teachers 
who participated in a six-week- summer institute on water pollution control, 
were compared with students of teachers who did not attend. The non-attending 
group included* teachers who had applied but were not selected to attend the 
institute-. Arialysis of covariance was used, a test of student achievement on 
knowledge of water pollution control as dependent variable, and DAT scores as 
covariate. Reported means for the classes of 'teachers v/hb attended were 
significantly higher than means for classes whose teachers- did not attend. 

f • . . 

. Method . ■ ■ « _ 

As .part of an independent evaluation of five NSF Comprehensive Teacher 
Training Projects funded by the "NSF, Gull ickson and .Welch (1972) carried 
out a large scale experimental design in which schools were systematically 
sampled within strata of urban-rural, geographic region, junior and senior 
'high school, and subject matter (science and mathematics,. were nested within . 
the comprehensive regions). Within each school selected, the principal 
was asked to select randomly one teacher from the science (or mathematics) 
faculty, who in turn was asked to select one. class at random from the. 



teacher 's class- load. Teachers and their selected. students were given a 
series of questionnaires,, including background questions for teachers and . 
the National Teacher Examinations in Physics-Chemistry-Science or Mathematics/" 
(Educatio^aUesting Service, 1970). Science students took the Test of 
Achievement in Science, or TAS (Lawrenz, 1972) which consists. of. items selected 
from the National Assessment of Educational Progress items in science; and 
mathematics students took the Mathematics Achievement Test-, abbreviated MAT, 
which was composed of items from The Natibnal--Longitudinal Study of . 
• Mathematical Abilities (NLSMA) pool iij mathematics (Sandman, 1972). Two 
forms were developed in science and in mathematics, one for junior high 

' level students (8th grade) ; and one for senior high level students (11th grade). 

' • ■ 2 ■ ■' • ' ■ 

. A tbtal of 3.46 teachers and classes 'participated in science, and 211 - . 

teachers and classes in mathematics . .Reliabilities reported for the tests 

^are given in Table 1. 

- . Table 1 

Reliabilities of Test Instruments 

Instrument . • Reliability 

NtE (Mathematics) Not Available . .94 • 

NTE.. (Science) Not Available - .90. • - 

TAS ' ' 

Senior High . 1921 . " .87(KR-20) 

Junior High . 981 .87( " ) 

' TAM . ; ^ " 

Senior-High. 1261.. .86( " ) 

Junior High 1A2A .92( • ) 



-''^P.egions • were centered in Wyoming ,.. South Dakota, and Mississippi for 
science; California and Indiana for mathematics. r' 

2 • ^ ■ ^ ■;. ^ _ ■ . • _ ' . ; . . 

Not all students took the achievement test. The teacher was given 
instructions for random divisibii of students; others took /attitude and process 
instrument^. Thus, class means are estimates for the class. 



■ Analysis ' ■ ■ * " 

As a way to estimate the contributory effect of NSF participation. . by 
teachers upon the achievement of their pupils, analysis of variance was 
chosen to partition, possible factors In student achievement. A regional - 
effect was Included as a fixed factor, since the generalizablllty to other 
regions Is uncertain. An urban-ruiral factor was considered but rejected, 
since for the science study no major urban centers were represented, in 
all three regions.. Only Denver, sampled in the Wyoming region, was a major 
urban center for science teachers. Similarly, the urban representation in 
the mathematics regions was small . (only Indianapolis). The majority of the 
schools sampled were in small towns and cities under 50,000 .population 
(81% in science, .91% in mathematics). 

A 'second factor considered was NSF participation by teachers. Partici- 
pation was defined as cumulative attendance at the following NSF-sponsored 
^.programs and institutes: AYI, ISI, SSI, and USI. Cooperative -.College and 
School . Science Programs (CCSS) were not considered since no information was 
gathered from the teachers about this type of NSF participation. No attempt 
was "made to look at each type of institute separately since the sample sizes 
for participation vs^. non-^partlclpatlon was so disparate. A frequency 
distribution of munber of institutes attended by science teacrhers yielded 
approximately equal percentages for no participation (36%) , one or two : 
institutes attended (36%), and three* to fourteen institutes attended (28%)'. 
These levels were then asslgned/as. !'N6|!^, /'Low**, and "High" leVels of 
participation in NSF-spohsored Institutes. The ^percentages of mathematics . 
teachers were 43% for the -'No" group., 29% .for the " "Low"' group, and 28% for . 
the ''lilgh" group, quite similar to the figures for the science teachers. 



A possible contaminant to examination of the effects of institute 
participation by teachers upon student achievement would be differential 
assignment of teachers to ability-grouped classes-. Teachers having attended 
institutes might bfe assigned to higher ability classes through seniority or 
for other reasons. This hypothesis was examined- by . testing the independence 
of NSF participation from the teachers' assessment of .the ability group, of 
the' class from which the achievement data were- dr^wh (high ability, average 

' • . ' . ■ - •■ v■■■^ . . . - ■ • 

ability, low ability^ and mixed ability grouping). Also tested within the 
senior high school science data was ..the independence of type of class (biology, 
chemistry, and physics) from NSF. participation. The chi-square statistic was 
used for each, test, and reisults are. presented in Table 2. ^1 chi-square 
statistics Were non-significant at p = .05, indicating independence of the 
distribution of teacher assignments by ability^ grouping, ^r subject matter in 
science, from NSF institute .participation. 

A-ijossible confounding of the analysis is due to the non-random selection 
of teachers for NSF. institutes. If "better" teachers (more academically capable) 
go to institutes, thema significant difference in student achievement among 
participation levels might be due to teacher subject matter competence. Accord- 
ingly, teacher achievement, as measured by the'NTE, was included in\ the analysis 
as a covariate, so that analysis of covariance with two fixed, crosiled -factors 
was performed on four ^sets of data: Test of Achievement in Science^(TAS) scores 
for' senior high students, TAS for junior high. Test of Achievement in Mathematics 
(TAM) scores for senior high, and TAM for junior* high. Similar results for both 
science and' mathematics would be interpreted" as validating the conclusions,, 
since the groups and tests are independent . . ... 

Disproportionate cell- sizes were encountered in/the science data,. so that' 
th^ design was made proportional to aid interpretation. This was done by 



Table .2 



Chi Square Values for Contingency Tables 
of NSF Institute Participation by 
- Ability Grouping or Subject Matter 



NSR Institute 
Participation 

(3 levels) 
by 



Teacher Rating 
of Class Ability 
(4 levels) 



2. Subject Matter 



Science 



Junior High 

= 1.12 

(df =6) 



(all 8'th grade) 
, science 



Mathematics 



Senior High 

7 ,85 
(df - 6) 



7.63 . 
(df = 4) 
(biology, chem- 



Junior High 




(all 8th 
grade) 



Senior High 

' 4:31 . . 
(df -6) 



(all 11th 
grade) j 



J o — ^ 1 

istry, physics?) ^nathematics mathematics 



randomly selecting out subjects in oversize cells, and increasing .sample size 
- in^ne case by including cell means as three dummy , subjects * scores (in the - 
science Miss^sippi junior high "High" participation group). Disproportionality 

was' not as great- In^ the mathematics study so that no sampling was performed* 

■ . ■ ■ \- ■ ■ ■ 

Cell sizes-, unadjusted means, and standard diviations are -given in. Table 3 . 

. Analysis of covariance was performed using the MULTIVARIANCE Program of 

Finn (1968). The results of the four , analyses are ^)resented in Table 4. . 



i - ■ Table. 3 ' 

Within-Cell Sample Size, Unadjusted Means, 

and Standard Deviations. for Four ANCOVA Problems 



Senior High Science Achievement 
- Region^ 



\ 



Mississippi 



South Dakota 



Wyoming 



Total' 



63 

'22.37 



• 6l 
■22.92. 



56 

24.51 



NO 


LCfW 


• HIGH 




Total' 


■ n=2 1 
-X=19.06 
s= 4.10 


'21 

. 18.83 
4.13 


14- 
18.41 

.2.94 




56 

18.81 


21 ■ 
21.21 

■ ; 'f\89 


21 ■ 
23.51 
3.98 


21 
- 24.71 
.3.94 




63 

23.15 


21 • 
. 26.85 
- , . 79 


21 ^ 
26.42 
3.60 

..■ 1 


. 21 
28.37 
3.08 




■ 63 
27.21 



Junior High Science Achievement " ' 



Region 

\ . 

Mississippi 



South Dakota 



Wyoming^ 



Tota 1 



NSF Participation 



. 42, ■ 
22;.20 



\ 20.35 
\'5.81 


1 


7 ■ , 

9.61 

3.24 


. 71 ■ 

18.00 
1.11 


1^4 ■ ■ 
2^.80. 
' - 3.17-' 

, \ 


14 . 
2l'.92 
4.13 


14 
: 25.28 
2.07 


■A- 

' ■ 3.5^ - 


- 14 

24.11 
' 2.73 


. 14 - 
23.59 
2 .34 



■•■ . 35 
'■22.33 



J5. 

.23.15 



Total 

28. 
19.58 



42 

23.67 



42 

23.39 



• .lOn J y four va I id ■ pbseryations werA. made for t!Us .ceil ; remaining observations 
•wdre ceJ 1 means . The s.tiindard deviatiirn is based on f our* observations . ; 
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Table 3 (Continued) 



Senior High Mathematics Achievement 



Region * 


■ NO' 


T nil 






'•' 18 


~ 17 


.20 


L»a X J. r OiT n la 


Z3 . 46 


25.68 ' 


25.64 




. 5.99 


3.62 


3.53- 














22 


19' . . 


Indiana . - 


• 22.26; 


25.73 . 


24.15 ■ 


. s ■ 

i 


5.61 \ 


- .\ 2.96 . 


.• 4.82 


\ '• 








\ Total 


34 ,\ 


'39 


39 ~" , 


• • ■ • K 


22.90 y 


.25.71 


24.91 



Total 
55 

24.94 



57 

24.23 • 



Junior High Mathematics Achievement 



Region : 


NO 


LOW 


HIGH- . 


. Total 


California . 


22 

19.11 

7:^.1 


15 

21.99 
6.97. 


11 • 
22.87 
6.07 


■ 48 : . , , 
20.87 1 


Indiana 


■\ 

33 \ 
21.00\ 
5.69 \ 

. ' \ 


■ 7 

17^52 
7>1^ 


11 

• 21.15 . 
5.03 


. .. n 

20.5-5 ■ • 


. Total 


55 

.20.24 


\ 22 ■ 
\ 20.57 


22 

22.01 . 





./ 

t , 




11 



ANCOVA- «Suinmar.y Table for t'our Analyses 



Science; . Senior High 



-•••■ Source " ' • . 


■ , -SS' ; ■ 


• df 


. '■MS::' 


F. 








A (Regloh) • • . ' : 
(NSF Par tlci pa ti&a) 


. 2 ,064^6 . ■ ■ 
V 79.8, 


■■2' ;' . 


.1>'P3:2.3 
' 39 . 9' • 


■ 71 


.29 , 
.76 


y < 
' < 


.01; 

.06^ 


A X.h ■ 


99.2 , 




' -24,8 


1 


.72: 




.15 


Error (within cell) 


2,49,0.6 


. 172 


■ 14.48*;;JJ; 










epvariate ^ 


r2=.006 . " 








.1- ■■ 


< 


.75 


Science: Junior High 






\ 1 








■'■jj,, ■ 


Source^ ^ T 


ss 


df 


MS 
.16$. 9 


,F 








A (Region) 


331.8 


2 - 


- 13 


.65 3^-' 


■< 


vOl 


B (NSF Participation) 


i3.i'-j 


■ ; 2 . 


' ■ ■ ■ ■ '6 \5. 




.54 




.59 


A X B '- /v: ■■ 


114.06 


4 


28.-5 ■' 


2 


.35 • 


< 


.06 


Error (within cell) 


1,239.95 


■ 102' 


12.1,6 


■ ^ 








Covariate 


r2=.002 


' 1 






.22 . 




.. 64 



.^p is referenced for 158 iif^rror, reduced by one for covariate and three 
^^tor dtimniy subjects • x ' 

2p £s referenced for 98 df, error , deduced by one £er epvariate and three 
.for dumnjy subjects'. \^ ^ ^ 

''^p is; a liberal estimate; positively ccrrel.ated sampt^^^isiziB^ error 
variance Implies smaller: p values.. ' 



0 

EKLC 



13 



. 12 



Table 4 (Continued)/'*' I 



Mathematics; ' Senidr High . 



\. Spurce 
v A (Region) 



SS 



v 



■16/60:.. 



• B- (NSF. Particlpatlbn) . ' ;. 157 , 94 ' 

A-X 'B. •/ : •• •:. : . ■ ■ ■■■581.68 ■■'■ 

/ ■ ' ■ . ■ ' ■ •■ /. 

Error- (within cell) : 2,107,94 



' . Covaria,l:e 



• - r2=,10 



■ H\ 

1 

. /I. 

105. 

■ ■ 1 



16^:60.^ ,83 ■.■■,■':>! 



5.5 
20, '08 



3,93 
.28 

1.15 



3 . 

:<.;37 
<,02 
<.76 

<.29' 



,. Mathematics; Junigr High 

Source ' ( , SS 
y "a (Region) . . 2,69- 

• ; ,B (NSF Participation) 47,9 
' • A X B ■ 168,8 . 

..Error (within cell) 3,767,9 • 

Covariate ' r2=', 008 



df ■ 


MS _ ■ 


. F ■ ' 


4' 

P 




.1 ' ■•■ 


2.69 


\ -.066 


<,.80 




! . ■ '■■ 
2., ' \ 


-. 23.95 . 


' :5.9. 


<.56 




2 " ■ 


. .84,4.0. ; 


■ 2.06 . 


^ <.:13 




92 

1. ■ 


40.96, 


, -.70 


■ <,41 





3 p is calculated for 104 df ^erroir, reduced by one for covariate. 

.' ' ■ ■ ■ ■ ■ ■.-'■■■ ' ' r ■ ' ■ ' 
■ is calculated for 91 df 't'rrqr, .reduced b.y one '-for covariate. 



i ■ 
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* : ' . Results • . • .' 

The marginal; means !of student achievement for. NSF participation show a " 



consistent trend in the direction of better .student performance. with increased 
teacher NSF participation for" all four analyses in Table 3 (note a minor , 
reversal between low and high participation for senior high science; both 
means are higher than that for ho participation)'. These means are essentially 



unaffected by adjustment for the "covariiate, since none of the regressions we^re 
significant at p =-^10/ Regional differences did occur, as noted by the ■ 
marginal means in Table 3. ^ .'-'^ .. * 

"^l The analysis of ^variance on ad justed means^alsd's^^ 

at,: the senior high level,— d iff e^^^^ octur oh the' NSF participation factor 

for both science and ma the Tia tics, while at the junior, ihigh level, the means . 
are not significantly diflBrent, although ordered' in the appropriate direction. 
Also for the . junior high analyses, marginally significant region by participa- 
tion interactions occur. These be.plo.tted but do? not seem , to . provide 
very enlightening results. . • . ' 

I /'The dispropdrtionality and unequal variances of the science data were 
examined- in order to adjust the estimated probability of occurrence of the 
obseryed F-statistics (see Glass, Peckham, and Saunders, 19.72). The-small 
variances with small cell sizes and large variances with large samples requires 
"either conducting a more liberal test (p = .10) or interpreting the observed 
probability as being liberal."^ In either case, the NSF partijdipa'tion factors 
are , significant for senior, high science and mathematics . Of int:ere$t within 
thes^e factors is where the differences lie. Two planned orthogonal contrasts 
were performed, as diagrammed below: (See Winer, 1971;* p. 172)'. - 



15 



14 



NSF Participation " 

Level ■/ m LO HI 

Contrast 1 . . -2 1 1 • . " ^ 

- Contrast- 2 * 0 -1 ' 1 * 

. "• • • • ' 'i • ^ '■ ' ■ 

■• ..; : . • / . - •', ' .'^ 

- • . Results are presented In Table -5. Among -the contrasts,- all. had .high", 

■significance except that of senior high mathematics for l^oy^^s . .^E^^ 

participation. The significant contrasts suggest. that teacher. attendance 

at /institutes is associated with higher student achievement than no attend-. 

ance, and that the students of teachers with high institute atterldance^\ 

perform better- than students of teachers who have only attended one or two 

institutes. . . - - ' 

Conclusions u . . ' 



The data presented here siippoxt the thesis, that participatiorf by senior 
High school science and mathematics teachers in NSF-sponsored .Institutes 
results in higher achievement scores by their pupils than for/ pupils of, ' 
teachers^ who do not attend^ The effects of differential teacher ability 
were removed by-- covariation; Similar trends were observed a't 'the junior high 
level, but 'the results are ►not. signif ican^t. That two ind^GMndeht data 

■■ ' ^ : ; . ' • ' ■ ' • ' ' ■ ' 

analyses should give such similar results is itself unusual, and the similarity 
is taken as support for the .thesis / The results ' also are/supported by the 
International" Study on Science in which . teacher training/variables were 
significant only at the senior high level. / • 

Significance at the senior high school and nonsighificance at the junior 
high level may be 'd^ie to the difference in subject matter taught and the 
population of students (ability, motivation, at.titude) between the two levels. 



Table 5: Planned Orthogonal l 
Contrasts for Significant NSF Participation Factors 



Achievement 



Subject ^ 


•Contrast 




■ df ; 


, ■ ;■ 


■ P 


Senior High Science . 


■ ' 1 . ' ■ 


7y.43. \ 


i;i72- 


■ .5.14 


2' 

. ■<.oi 


.->•■■ . , 




74.95 ' 


..1,172 


. 5.18 . 


< i.Ol 


Senior High Mathematics 


1 


137.53 


1,105 


6.85 


■ :<'0i 




2 


. ,12.48 


1,105 ' 


.62 


<.5 . 



■ MSe;is referenced; from Tablec A. -''^ f 

■ The. actual va.lue may be lower because of disproportipnality and unequal 
-cell variances; r\.o research 'data are availab|e on the robustness of^ Planned 

Orthogonal Contrasts for ..violations: of assumptions. "* 



Thus, the functions played by. institutes, may operate more: effectively, with 

the more select senior high school students who elect bialogy, chemistry, 

and physics. This group is not identical in composition with the junior 

high science students, ail of whom must take science, generally. 

An \alternativewexplanation for the results may be a. selection bias 
; -v. - \ ' ■ ^ . • . -. . •■■ ■ ■ 

/between those who attend* arid those. who do not, attend based upon motivation 

arid attitude. These criteria were not used to select individuals- for NSF 

•institutes but may, in fact., operate through self-selection of teachers 



who- apply .-^ As with.; teacher achievement, however , little research data has 
' • ■ ^« ■ • ■ ■ , . • ■ 

shown conclusively that teacher . attitude is a determinant fO'r . student 

achievement. ■ ■ ' 
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